An Asymptotically Optimal UCB Policy for Uniform Bandits of Unknown Support

Consider the problem of a controller sampling sequentially from a finite number of N ≥ 2 populations, specified by random variables X k, i = 1, . . . , N, and k = 1, 2, . . .; where X i k denotes the outcome from population i the k time it is sampled. It is assumed that for each fixed i, {X k}k≥1 is a sequence of i.i.d. uniform random variables over some interval [ai, bi], with the support (i.e., ai, bi) unknown to the controller. The objective is to have a policy π for deciding, based on available data, from which of the N populations to sample from at any time n = 1, 2, . . . so as to maximize the expected sum of outcomes of n samples or equivalently to minimize the regret due to lack on information of the parameters {ai} and {bi}. In this paper, we present a simple UCB-type policy that is asymptotically optimal. Additionally, finite horizon regret bounds are given.